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Abstract — This paper presents a new combination scheme for 
reducing the number of focal elements to manipulate in order to 
reduce the complexity of the combination process in the multi- 
class framework. The basic idea consists in using of p sources of 
information involved in the global scheme providing p kinds of 
complementary information to feed each set of p one class 
support vector machine classifiers independently of each other, 
which are designed for detecting the outliers of the same target 
class, then, the outputs issued from this set of classifiers are 
combined through the plausible and paradoxical reasoning 
theory for each target class. The main objective of this approach 
is to render calibrated outputs even when less complementary 
responses are encountered. An inspired version of Appriou’s 
model for estimating the generalized basic belief assignments is 
presented in this paper. The proposed methodology allows 
decomposing a //-class problem into a series of //-combination, 
while providing //-calibrated outputs into the multi-class 
framework. The effectiveness of the proposed combination 
scheme with proportional conflict redistribution algorithm is 
validated on digit recognition application and is compared with 
existing statistical, learning, and evidence theory based 
combination algorithms. 

Keywords — Conflict management; Dezert-Smarandache theory; 
Handwriting digit recognition; Support Vector Machines. 

I. Introduction 

Nowadays a large number of classifiers and methods of 
generating features is developed in various application areas of 
pattern recognition [1], [2]. Nevertheless, it failed to underline 
the incontestable superiority of a method over another in both 
steps of generating features and classification. Rather than 
trying to optimize a single classifier by choosing the best 
features for a given problem, researchers found more 
interesting to combine the recognition methods [2]. Indeed, the 
combination of classifiers allows exploiting the redundant and 
complementary nature of the responses issued from different 
classifiers. 

Researchers have proposed an approaches for combining 
classifiers increasingly numerous and varied, which led the 
development of several schemes in order to treat data in 
different ways [2]. Generally, three approaches for combining 
classifiers can be considered: parallel approach, sequential 
approach and hybrid approach [2]. Furthermore, these ones can 



be performed at a class level, at a rank level, or at a measure 
level [3], 

However, with the existence of the constraints 
corresponding to the joint use of classifiers and methods of 
generating features, an appropriate operating method using 
mathematical approaches is needed, which takes into account 
two notions: uncertainty and imprecision of the responses of 
classifiers. In general, the most theoretical advances which 
have been devoted to the theory of probabilities are able to 
represent the uncertain knowledge but are unable to model 
easily the information which is imprecise, incomplete, or not 
totally reliable. Moreover, they often lead to confuse both 
concepts of uncertainty and imprecision with the probability 
measure. Therefore, a new original theories dealing with 
uncertainty and imprecise information have been introduced, 
such as the fuzzy set theory [4], evidence theory [5], possibility 
theory [6] and, very recently, the theory of plausible and 
paradoxical reasoning in [7], [8], [9]. The Dezert-Smarandache 
theory (DSmT) of plausible and paradoxical reasoning was 
elaborate by Jean Dezert and Florentin Smarandache for 
dealing with imprecise, uncertain and paradoxical sources of 
information. Thus, the main objective of the DSmT was to 
introduce combination rules that would allow to correctly 
combining evidences issued from different information 
sources, even in presence of conflicts between sources or in 
presence of constraints corresponding to an appropriate model 
(i.e. free or hybrid DSm models [7]). The DSmT is justified in 
many kinds of applications [7], [8], [9]. 

Indeed, the DSmT has a feasible computational complexity 
for industrial uses which are considered as problems of small 
dimension [10], [11]. In contrast, the extension of this theory 
into the multi-class framework has the problem of their 
applicability in view of the high computational complexity. 
This is closely related with the number of elements to be 
processed in the framework of this theory, which follows the 
sequence of Dedekind’s numbers [12]. Try to use the free DSm 
model, considering the set of all subsets of the original classes 
(but under the union and the intersection operators), is not easy 
and becomes untractable for more than 6 elements in the 
discernment space [13]. 

In this paper, we propose an effective combination scheme 
of one-class classifiers in a general belief function framework 
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by incorporating an intelligent learning technique for reducing 
the number of focal elements. This allows us to reduce 
drastically the computational complexity of the combination 
process and to extend specially the applicability of DSmT into 
the multi-class classification framework. Indeed, the objective 
of this work is neither to choose the kind of the one-class 
classifier, but only to illustrate from a practical application the 
advantage of this new combination scheme for the real-time 
implementation purpose. 

In Section II, this paper will deal with the related works. 
Theoretic formulation of Proportional Conflict Redistribution 
(PCR6) combination rule and the way it can be extended for 
solving the multi-class classification problem are presented in 
Section III. We give in Section IV a multi-class classification 
scheme based on belief function theories. The database of the 
isolated handwritten digits, methods used for generating 
features and algorithm used for OC-SVM models validation are 
described in Section V. The experimental and statistical results 
are summarized in Section VI. 

II. Related Works 

Dezert and Smarandache [14] proposed a first work for 
ordering all elements generated using the free DSm model for 
matrix calculus such as [15], [16] made in DST framework. 
But, this proposition has limitations since in practical 
applications it is better to only manipulate the focal elements 

[17] , [18], [19], [3], 

Hence, few works have been focused already on the 
computational complexity of the combination algorithms 
formulated in DSmT framework. Djiknavorian and Grenier 

[18] showed that there’s a way to avoid the high level of 
complexity of DSm hybrid (DSmH) combination algorithm by 
designing a such code that can perform a complete DSmH 
combination in very short period of time. However, even if 
they have obtained an optimal process of evaluating DSmH 
algorithm, first some parts of their code are really not 
optimized and second it has been developed only for a dynamic 
fusion. Martin [20] further proposed a practical codification of 
the focal elements which gives only one integer number to each 
part of the Venn diagram representing the discernment space. 
Contrary to the Smarandache’s codification [13] used in [21] 
and the proposed codes in [18], author thinks that the 
constraints given by the application must be integrated directly 
in the codification of the focal elements for getting a reduced 
discernment space. Therefore, this codification can drastically 
reduce the number of possible focal elements and so the 
complexity of the DST as well as the DSmT frameworks. A 
disadvantage of this codification is that the complexity 
increases drastically with the number of combined sources 
especially when dealing with a problem in the multi-class 
framework. To address this issue, Li et al. [22] proposed a 
criterion called evidence supporting measure of similarity 
(ESMS), which consists in selecting, among all sources 
available, only a subset of sources of evidence in order to 
reduce the complexity of the combination process, but this 
criterion has been justified for only a two-class problem. 
However, the complexity of reducing both the number of 
combined sources and the size of the discernment space are 
research challenges that still need to be addressed. 



III. Theoretic Formulation of PCR6 Combination 
Rule and Application to Multi-class Classification 

We denote by p sources S l ,S 2 ,-..,S p of information 
corresponding to measures issued from p one-class classifiers, 
which are involved to deal with a one-class problem. For each 
source, two classes Q, and 0 j are considered being the target 
class and their complementary’ class, respectively. 

To generate the combined masses from all the p one -class 
classifiers, a set of elements is defined namely 
- {H 1 ,...,H Cflj .( G ( i ))|, which attributes for each element 

A, (/ = 1 , . . . , Car (g ■ ' 1 )) a “belief’ value m k (A - ) associated to 
each source S k taking the value in the range [0, l] and 

verifying ^°" <G ) m k {Aj)= 1, such that Carfa is the 

cardinal of the set and ke {l, 2, ...,p] defines the index of 
the corresponding source. 

According the finite set of hypotheses, a combination rule 
is then performed on elementary masses for generating partial 
masses from all the p sources as follows: 

m { f\A)=[m x @--@m p \{A),A& G (i) , (1) 

where my(A) is the partial mass associated to the elementary 
or compound hypothesis A of the combined sources and 

Z Cflr(G^) / \ 

. =1 ™ c \Aj)= 1 . 

Hence, the choice of an appropriate combination rule 
depends on the set of predefined hypotheses. Example of such 
approaches is Proportional Conflict Redistribution (PCR6) rule 
based on DSmT. 

A. Combination Rule Based On the DSmT 

Let © = {#!, # 2 the discernment space of the multi- 
class classification problem under consideration having n 
exhaustive elementary’ hypotheses which are not 

necessarily mutually exclusive in DSmT. The main concept of 
the DSmT is to distribute basic belief assignment of certainty 
over all the composite propositions built from elements of 
subset of 0, namely 0^ = {#,,#,}, with u (Union) and n 
(Intersection) operators instead of making this distribution over 
the elementary or union hypothesis only. Therefore, the hyper- 

powerset for two hypotheses (classes) belonging to 0 (,i is 
defined as = D 0 *'* =\d,Q i ,Q j ,6 i u0,-,0,- n#,}. The DSmT 
uses generalized basic belief mass, also known as the 
generalized basic belief assignment (gbba) computed on hyper- 

powerset of 0^, which is defined as: m k (&) = 0 and 
m k (&,)+ m k fa )+ m k fa u 6, )+ m k fa n<9,)= I , such that 
m k [0 j yj0 ! ) defines the mass of the ignorance, m k fa i n0 i ) 
represents the mass of the conflict (or paradoxical information), 
and ke {l, 2, The way the conflicting mass is 
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redistributed yields to several versions of a Proportional 
Conflict Redistribution (PCR) rules [23], [19]. Form PCR1 to 
PCR2, PCR3, PCR4, PCR5 one increases the complexity of the 
rules and also the exactitude of the redistribution of conflicting 
masses. The combination rule (PCR5) proposed by [23] for two 
sources is mathematically one of the best for the proportional 
redistribution of the conflict applicable in the context of the 
DSmT. Martin and Osswald have proposed the following 
alternative rule to PCR5 for combining more than two sources 
altogether (i.e. p upper then 3). This new rule denoted PCR6 
does not follow back on the track of conjunctive rule as PCR5 
general formula does, but it gets better intuitive results. For 
p = 2 PCR5 and PCR6 coincide. The combined partial gbba 

m pcR6 obtained from p generalized belief assignments 
rrii (.), . . .,m p (.) by means of the PCR6 rule [19] is defined as: 



m 



- m $ CR6 (A) 



Where 



m *( A )+ 

Yj m k( A T 



k = 1 



if Ae O, 



y,L k otherwise. 

L 0 ,., Wl) M G *r 



( 2 ) 



p — 1 

W m °k(j)^°k(j)) 



L k =- 



j = i 



p — t ’ 

) + ^ m o k O') 0) ) 

7=1 



( 3 ) 



O = {O M ,0} is the set of all relatively and absolutely empty 
elements, O M is the set of all elements of which have 
been forced to be empty in the hybrid model M defined by 
the exhaustive and exclusive constraints, 0 is the empty set, 

p- 1 , , 

the denominator m k (a) + y m (jdi) Vo A /) ) is different to zero, 
7=1 

and where c 7 k counts from 1 to p avoiding k, i.e.: 

j if j -< k. 



U + l if j>k. 



( 4 ) 



Flere, ni[ > ' l {A) corresponds to the classical DSm rule on the 
free Dsm model [24], which is defined as: 

f0 if 4 = 0, 



«A ) U) = 



f' 

otherwise. 



( 5 ) 



B l , B 2 ,..., B p eG [l > k=l 
B l r \ B 2 r \... nB=A 



B. Effective Combination Scheme of One-Class Classifiers 
For the computation of the global combined mass m c (.) , 

the direct use of PCR6 combination rule on D @ yields the 
computation cost that increases drastically with n and even 
may be computationally prohibitive, especially when we have a 
huge number of elements belonging to D e when dealing with 
gbba's within the DSmT framework (i.e. Card [d 0 )= d(n) , 
such that d(n) is the Dedekind’s number of n ). Flowever, for 
our multi-class problem, because of a special feature that the 
separation of data according to One Against All (OAA) 
approach [25] of such classifier, we can render the data highly 
unbalanced for each two-class problem. Flence, the need to use 
an one-class classifier which is able to distinguish the samples 
of the target class 0 , from other outliers belonging to its 

complementary class , i = l,...,n. From this principle, we 
propose a combination scheme which allows decomposing a 
n -class problem into a series of n -combination, whose 
reasoning for each combination is performed from the subset 
0^,i = 1,...,« , instead of the reference space 0. Flence, we 
propose a combination scheme which uses a complementary 
features captured by the different sources of information 
S k ,k =\,...,p , from the input probe data to feed the one-class 
classifiers OC-C ik ,k =l,...,p , that operate independently of 
each other for each target class 6 t ,i = \,...,n , and then the 
partial opinions (i.e. transformed measures) provided from 
these classifiers will be combined all together through an 
appropriate rule in the subset G^\i = l,...,n . Finally, all the n 
partial combined masses will be incorporated in a unique 
module for the task of decision making. Table I gives a 
comparison of Card (2®) and Card[p e ) with respect to 

n . \ 

Card{F ) = y CardyG^’J, which defines the cardinal of set of 

i = 1 

all focal elements F = |g^,/ = 1, ...,«} obtained according to 
our computing method within DSmT framework, as follows: 

TABLE I 



Cardinality of Combination Space 



Card(®) = n 


Card[l & ) 


Card[o Q ) 


Card(F) 


2 


4 


5 


5 


3 


8 


19 


11 


4 


16 


167 


14 


5 


32 


7580 


17 


6 


64 


7828353 


20 



In this way, we can reduce drastically, within DSmT 
framework, the number of focal elements from 

Card (d 0 )= d(n) to Card (f) = 3 x n + 2, V« > 3 . 
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IV. Multi-Class Classification Scheme Based On 
Belief Function Theories 

The proposed multi-class classification scheme 
incorporates mainly four modules: i) one-class support vector 
machine (OC-SVM) classification, ii) transformation of the 
normalized OC-SVM outputs into belief assignments using 
estimation technique based on a modified version of Appriou's 
model, iii) combination of masses through an algorithm based 
on belief function theories and iv) decision making. 

A. Classification Based On OC-SVM 

In the proposed combination scheme, we take the path of 
OC-SVMs, enabling us to incorporate an intelligent learning 
technique to efficiently avoid both the closed set and good 
distribution assumptions in the multi-class classification 
framework. In the following, we briefly review the concept 
learning with one-class SVM. 

Review of OC-SVM: Scholkopf et al. [26] proposed OC- 
SVM classifier by modifying the standard support vector 
machines initially introduced by Vapnik [27]. The pattern 
classification approach using OC-SVM has been successfully 
used for many applications as biometric verification [28], [29]. 
This classifier is an unsupervised learning algorithm, which 
only requires the learning of the target class samples. In fact, 
the pattern classification through OC-SVM consists of defining 
a boundary around the target class, such that it accepts as many 
of the target samples as possible, while minimizing the chance 
of accepting outliers. 

The concept of the OC-SVM seeks to find an hyper sphere 
in which the most of learning data are included into a minimum 
volume. More specifically, the objective of the OC-SVM is to 
estimate a function f oc (x ) that encloses the most of learning 

data into a hyper sphere R x = {x e R rf ,f oc (x) >- o} with a 
minimum volume where d is the size of feature vector [26]. 
Hence, the decision function foci x ) * s given as [26]: 

Sv 

foe M = K ( x ’ x j (6) 

j 

where Sv is the number of support vectors x • form the 
training dataset, a j are Lagrange multipliers, such that 

0 < a : < , m is the cardinal of training dataset, v is the 

J vm 

percentage of data considered as outliers, p defines the 
distance of the hyper sphere from the origin, and defines 
the OC-SVM kernel that allows projecting data from the 
original space to the feature space [30]. 

A pattern x is then accepted when f oc (x ) >- 0 . Otherwise, 
it is rejected. Various kernel functions can be used as 
polynomial. Radial Basis Function (RBF) or multilayer 
perception [27]. Generally, the RBF kernel is used for its better 
performance, which is defined as: 




where y is the kernel parameter. 

In the following, we show how the OC-SVM based concept 
learning can be extended to construct multi-class OC-SVM 
with multiple hyper spheres. 

Extension of OC-SVM for Constructing Multi-Class OC- 
SVM: Basically, the OC-SVM classifier have been conceived 
to deal with an one-class classification problem [26]. Their 
extension to multi-class scenario may provide uncalibrated 
outputs for some classifiers. In this paper, we use a sigmoid 
transformation for mapping the reassigned output, using 
logarithmic function, of different OC-SVM classifiers to 
probabilities as follows: 



pM!x)= 



yz k 

1 + exp(- g ik (x)) ’ 



( 8 ) 



n - 1 

where Z k = ^[l + exp(-g I/t (.t))] _1 represent normalization 

1=0 

factors that are introduced in the probabilistic framework in 
order to respect the normality condition (i.e. ^PM/x)=D, 

S,eQ 

P k are the posterior probabilities issued from the source of 
information S k ,k = l,...,p . Thus, the term g ik (x) represents 
the reassigned output of the z — th OC-SVM classifier, 
i = 1 , . . ., n , which is defined for a given pattern x as: 

Sik( x )~ “log 

Sv^and p ik are the number of support vectors and the 
distance of the hyper sphere from the origin for each 
OC-SVM ;yt which is trained with samples of the /-th class 
0j,i = l,...,n , provided by the source of information 
S k ,k =\,...,p respectively. 

In the multi-class classification framework, OC-SVM 
classifier is extended. Therefore, the posterior probability 
P k [O t /x ) of any target class 0 i ,i = l,...,n , of the frame 
0 = {#i, #2 -••••’$«} can be directly obtained according the 
equation (8). Finally, the maximum likelihood (ML) test is 
used for decision making as follows: 

xe 6 t if P [d l /x) = maxjp .. [Oj jx\\ <j< n\ (10) 

where x is the pattern test characterized by the source of 
information S k ,k — \,...,p . 



Sv ik 

I 



t XjK(xp 






+ l°g(Ait )> (9) 



B. Estimation of Masses 

In this paper, the mass functions gbba of evidence m k (.) 
issued from the k - th source of information S k ,k = 1 , ...,p , 
are estimated using an inspired version of Appriou's model, 
which is initially defined for two classes [31]. Therefore, the 
modified version of Appriou’s model in DSmT framework 
over = jo, u#,-,#,- n 0,-}, i = is given as: 
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»;,(©)= 0, 


(11) 


, , (l -A)7>, (»,/*) 

m * [ffi) -(i + £)(i + p k (0t/x)y 


(12) 


m (#")- Pi 

mk ™- l + etl + PMlx))' 


(13) 




(14) 


m k (0,r<0)=^-, 


(15) 



where £ > 0 is a tuning parameter, and jf j is the sum of false 
accepted rates (FAR) made by the OC - SVM, k , k = 1 , ...,p , 
classifiers, which are trained with p sources of information, 
respectively. Furthermore, /^/(l + f) is used to quantify the 
belief for conflicting region, and e/{[ + e ) is used to quantify 
the belief that the pattern x belong to the subset 
= . Therefore, the value of £ is fixed here to 

0 . 001 . 

C. Combination of Masses 

In order to manage the conflict generated from p sources 
of information (i.e. S k ,k = l,...,p ), the global combined 
masses are computed over the subsets G^\i = , as 

follows: 

m c (A) = -Xm^(A) 

< n (16) 

with Ae G^-‘\i = 1 

where 1 /n represents normalization factor that is introduced in 
DSmT framework in order to respect the normality condition 
of masses over the set of all focal 

elements F = {g^,/ = 1, ...,«} (i.e. ^m c (a) = 1 ), m$ CR6 () is 

AeF 

the partial combined mass using the PCR6 combination rule, 
such that ni'f) CR6 (a) = (w, © • • • © m ]{A) , and © represents the 

combination operator, which is composed of both conjunctive 
and redistribution terms of the PCR6 rule, when dealing with 
DSmT framework. 

D. Decision Rale 

Combination of evidences using the proposed combination 
scheme yields the combined belief and a decision making is 
made using the statistical classification technique. First, the 
combined beliefs are converted into probability measure using 
a probabilistic transformation, called DSmP, that maps a belief 
measure to a subjective probability measure [32] defined as: 

0 if ^ e 0, 

DSmP e (A i )=^!H mc ( Aj ) otherwise, (17) 

AjeF *} 

where 



Wj= (18) 

A k (zA/nAj 

Cm (A )=1 

Tj= '£ / m c {A k )+eC M (A J ). (19) 

4= f- 
CmM = 1 

E > 0 is a tuning parameter and F corresponds to the set of 
all focal elements including eventually all the integrity 

— M 

constraints (if any, i.e. 3ie {l n <9, =0) of the model 

M (_F = |g^,V;g {l , Gt r\6[ = oj for Shafer’s model 

and 7 7 = |g^,V;g {l , =dj c\9j ^0j for all the 

paradoxical hypotheses); C M {A k ) denotes the DSm cardinal 
of the set A k [14]. 

In the context of some particular multi-class classification 
problems, the simple classes Q, are truly exclusive and 
Shafer’s model is adopted. Therefore, the DSmP e (0t) 
probability of any element $,,/ = 1 ,...,« , of the frame 
© = {O\,02^--A„} can be directly obtained according the 
following equation: 



DSmP e (0j ) = 




In this manner, the combined belief assignment is transformed 
into a probability measure so that the statistical classification 
approach is applied for computing the final decision. Finally, 
the DSmP -based maximum likelihood (ML) test is used for 
decision making as follows: 

xe 0j if DSmP e (0j ) = max [DSm P e ( <9 ; ), 0 < j < n\, (21) 

where x is the pattern test characterized by p sources of 
information S k ,k =l,...,p , and e is fixed to 0.001 in the 
decision measure given by (17). 

V. Database and Algorithms Used for Validation 

The proposed OC-SVM classifiers are trained using 
different methods of generating features on a database of the 
isolated handwritten digits. In this section, we briefly describe 
the database, the methods used for generating features and the 
algorithm used for validation of OC-SVM models. 

A. Database Description and Performance Criteria 

To validate the proposed combination scheme, the well- 
known US Postal Service (USPS) database is used for 
handwriting recognition task. This database contains 
normalized grey-level handwritten digit images of 10 numeral 
classes, extracted from US postal envelopes. All images are 
segmented and normalized to a size of 16x16 pixels. There 
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are 7291 training data and 2007 test data where some of them 
are corrupted and difficult to classify correctly. For evaluating 
the performances of the combination scheme, a popular rates 
are considered, which are the Recognition Rate (RR) for each 
class and Mean Recognition Rate (MRR) for all classes. 

B. Methods Used for Generating Features 

The objective of the features generation step is to underline 
the relevant information that initially exists in the raw data. 
Thus, an appropriate choice of the descriptor improves 
significantly the accuracy of the combination scheme. In this 
study, we use a collection of popular feature generation 
methods, which can be categorized into background features, 
foreground features, geometric features [3]. 

C. Algorithm Used for Validation of OC-SVM Models 

The OC-SVM model is produced for each class according 
the used descriptor. Hence, the training dataset is partitioned 
into ten subsets of samples: each one is used as a learning 
subset to learn the corresponding OC-SVM classifier that 
operates independently of other. Let N different values of the 
RBF parameter y t , sorted in increasing order, such that 
i = 0,1,...,N — 1 . The hyper parameters of each OC-SVM 
model are tuned during the validation phase using the 
corresponding training subset of samples. In this work, we 
have allowed until 10% of error on the training dataset (i.e. 
percentage of training data considered as outliers V < 0. 1 ) for 
each target class. Hence, the j - th OC-SVM model is 

generated under constraint HR . -< i/x 1 00 , such that ER ; is 
the corresponding error rate computed during the validation 
phase. Indeed, the selection of the optimal value yj : ‘J t , of the 

RBF parameter during the validation phase is performed for 
each target class 0 i ,i = 0,...,9, on the set of all models 
fulfilling the last condition, using the maximum of the number 
of support vectors Sv criteria. Consequently, higher the number 
of support vectors is, the better the information is 
representative for each class. 

VI. Experimental Results 

The effectiveness of the proposed combination scheme is 
demonstrated experimentally by evaluating the recognition 
performance on all isolated handwritten digits from the test 
dataset. We perform experiments to select a subset of global 
complementary sources of information using the proposed 
extension of OC-SVM into multi-class (MC) classification 
framework, namely MC-OC-SVM, and then the proposed 
combination scheme is evaluated in belief function theories 
framework. 

A. Performance Evaluation of the Proposed Descriptors 

In these experiments, we compute during the test phase the 
recognition rate MRR of the MC-OC-SVM classifier using 
Geometric Features (GF), Foreground Features (FF), 
Background Features (BF), and the descriptors which result 
from a concatenation between at least two simple descriptors 
such as (BF,FF), (BF,GF), (FF,GF), and the (BF,FF,GF) 



descriptor. Indeed, the experiment has shown that the 
appropriate choice of both descriptors and concatenation in 
order to represent each digit class in the feature generation step 
provides an interesting recognition performance. In Table II, 
the MRR vary from one descriptor to another, and the MRR of 
concatenated descriptors are relatively high compared to those 
of simple descriptors. 

As we can see, it is difficult to improve the recognition 
performance by a concatenation of features since most of the 
time the combined descriptors does not take into account the 
complementary nature of features, which can be exist between 
both descriptors. Hence, we choose among all descriptors 
available (i.e. see Table II) only those for which the 
corresponding MC-OC-SVM classifiers could attain an 
improvement in the recognition performance. Indeed, BF, FF 
and GF-based descriptors yield respectively in the experiments 
(c), (b) and (a) a MRR of 89.50%, 83.75% and 78.90%. When 
using (FF,GF)-based descriptor in the experiment (f), we obtain 
a significant improvement in the recognition performance of 
the MC-OC-SVM classifier from 83.75% until 87.59%. 
Further, an important gain of 4.31% in the recognition 
performance, where MRR = 91.71%, is obtained in the 
experiment (g) when the BF-based descriptor is concatenated 
to the (FF,GF)-based descriptor to get a new (BF,FF,GF)-based 
descriptor. 

Hence, in the following section we use the three descriptors 
BF, (FF,GF) and (BF,FF,GF) as global sources of information 
to feed the OC-SVM classifier of each target class. This allows 
us to evaluate the recognition performance of the proposed 
combination scheme and to better exploit the complementary 
nature, which is obtained from these descriptors. In this way, it 
is possible to improve the recognition performance when the 
concatenation of descriptors can fail to provide the correct 
solution for some specific handwritten digit recognition 
problems. 

B. Performance Evaluation of the Proposed Combination 

Scheme 

In these experiments, we evaluate the recognition 
performance of the proposed combination scheme using sum, 
DS and PCR6 rules. In fact, this combination scheme allows to 
exploit the complementary nature issued from the three sources 
of information 5j = BF,S 2 = (FF,GF),53 = (BF,FF,GF) , and 
manage the conflict provided from the outputs of OC-SVM 
classifiers of each target class. 

The proposed combination scheme, which uses ten 
combinations (i.e. a combination per target class), consists to 
measure ten values of conflict K CI ,1,...,9, for each isolated 
handwritten digit from the test dataset. In the context of 
recognition of isolated handwritten digits, the conflicting 
regions of each handwritten digit test are modeled by the ten 

paradoxical classes 0 i n0 i ,i = O,...,9 . Consequently, we 

introduce an integrity constraint O i C\Q i ,i = 0,...,9, for each 
corresponding combination. In Table III, each line presents the 
FAR error computed for each class of handwritten digits using 
the three sources of information S 1 ,S 2 and S 3 . Indeed, the 
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TABLE II 

Recognition Rates of The MC-OC-SVM Classifier Using Different Methods of Generating Features 



Descriptor 


Simple Descriptor 


Concatenated Descriptor | 


(a) GF 


(b) FF 


(c) BF 


(d) (BF,FF) 


(e) (BF,GF) 


(f) (FF,GF) 


(g) (BF,FF,GF) 


MRR (%) 


78.90 


83.75 


89.50 


90.70 


91.90 


87.59 


91.71 



proposed OC-SVM models are optimized and trained using the 
training dataset to compute, during the validation phase, the 
error parameter /?, ,/ = 0, ... ,9 shown in the last column of this 
table, as the sum of FAR errors which is subsequently used for 
each combination. As we can see here, the 8-th OC-SVM 
classifier provides the higher value of error of FAR, where 
/3 S = 6.69% , for each source of information. 



TABLE III 

FAR Errors Provided By The Tree Sources of 
Information For Each Class 



Class 


FAR (%) 


Pi (%) 


Source 1 


Source 2 


Source 3 


0 


0.07 


0.00 


0.00 


0.07 


1 


0.10 


0.06 


0.06 


0.22 


2 


0.67 


1.14 


0.18 


1.99 


3 


0.24 


0.24 


0.11 


0.59 


4 


0.02 


0.00 


0.00 


0.02 


5 


0.60 


0.64 


0.21 


1.45 


6 


0.02 


0.33 


0.14 


0.49 


7 


0.03 


0.00 


0.00 


0.03 


8 


2.48 


2.88 


1.33 


6.69 


9 


0.08 


0.14 


0.06 


0.28 



For better comparison, recognition results corresponding to 
the combination of the three sources S 1 ,S 2 and S 3 by sum, 
DS and PCR6 rules are respectively given in Table IV. The 
proposed combination scheme using the DS rule yields a MRR 
of 92.76% corresponding to an improvement of 1.05%. While 
sum rule decreases the MRR to 92.68%. This is due to the 
direct estimation technique of masses which assigns the 
confidences only to the simple classes in probabilistic theory 
framework. Hence, the sum rule couldn’t handle managing 
correctly the conflict generated from the three sources. 
Furthermore, the experimental results show that the DS rule is 
not able to handle most of the conflicting cases between the 
three sources. Hence, the DST is not appropriate to solve our 
problem of handwritten digit recognition into the multi-class 
classification framework. Indeed, the use of DS rule in the 
combination scheme allows redistributing the beliefs through a 
simple normalization by (l — K CI ,z = 0,1,... ,9) in the 

combination process of masses. However, when responses of 
OC-SVM classifiers are less complementary, both sum and DS 
rules do not provide reliable decision. In Table IV, an 
improvement of RR when using PCR6 rule compared to both 
sum and DS rules were obtained for all classes of handwritten 
digits except those of 6 ] , 0 2 and 0 S . This is because there are 
some digits belonging to theses classes which are wrongly 
characterized by the combined sources >Sj , S 2 and S 3 . In other 
words, the PCR6 combination based rule is not reliable when 
the complementary information provided from the sources of 
information is wrongly preserved. In order to get a higher 
MRR, the combined sources of information should provide 
complementary information. The proposed combination 
scheme with PCR6 rule yields the best MRR of 95.03% when 



TABLE IV 



Comparison of The Recognition Performance of Sum, DS and 
PCR6 Combination Rules Using Three Sources (Si, S 2 , S 3 ) 



Class 


Sources 


Combination Rules | 


S, 


S 2 


s 3 


Sum 


DS 


PCR6 


0 


81.05 


80.22 


86.07 


86.35 


86.35 


94.15 


1 


85.55 


94.67 


96.57 


95.06 


95.05 


96.20 


2 


92.78 


92.78 


96.39 


95.36 


95.87 


94.84 


3 


81.09 


75.00 


85.97 


89.02 


89.63 


92.07 


4 


94.44 


87.87 


91.91 


93.94 


93.43 


95.95 


5 


91.81 


84.37 


89.37 


91.87 


91.87 


95.00 


6 


92.30 


91.12 


94.67 


94.08 


93.49 


97.63 


7 


91.09 


91.78 


91.78 


93.15 


93.84 


93.83 


8 


93.29 


94.51 


95.73 


98.17 


98.17 


95.12 


9 


91.52 


83.61 


88.70 


89.83 


89.83 


95.48 


MRR (%) 


89.50 


87.59 


91.71 


92.68 


92.76 


95.03 



combining the three sources S 1 ,S 2 and S 3 all together. 
Indeed, the PCR6 rule allows an efficient redistribution of the 
partial conflicting mass only to the elements involved in the 
partial conflict (i.e. the target class 6 t and their complementary 

class 8 i , / = 0,1,..., 9). After redistribution, the combined mass 
is transformed into the DSm probability and the DSmP -based 
ML test is used for decision making. Finally, the proposed 
combination scheme using PCR6 rule in DSmT framework is 
the most stable across all experiments whereas recognition 
rates pertaining to DS combination rule vary significantly. 

VII. Conclusion And Future Work 

In this paper, an effective combination scheme of one-class 
classifiers in a general belief function framework has been 
proposed. The OC-SVM classifiers can be incorporated as an 
intelligent learning technique for reducing the number of focal 
elements. This scheme consists in using of a subset of global 
complementary sources of information to feed the OC-SVM 
classifiers corresponding to each target class, which allows 
decomposing a « -class problem into a series of /7-combination, 
while providing /7-calibrated outputs into the multi-class 
framework. Therefore, this allows us to reduce drastically the 
computational complexity of the combination process and to 
extend specially the applicability of DSmT into the multi-class 
classification framework. Experimental results show that the 
proposed combination scheme with PCR6 rule yields the best 
performance on the handwritten digit recognition application 
compared to the sum rule and DS rule even when the 
individual MC-OC-SVM multi-classifications provide 
uncalibrated outputs. 

In continuation to the present work, the next objectives 
consist to adapt the use of the evidence supporting measure of 
similarity (ESMS) criteria to select complementary sources of 
information for each target class using the same proposed 
combination scheme in order to attempt to improve the RR and 
MRR. 
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